Handling Moving Objects and Over-Exposed Regions in Non-Blind Multi-Image Deconvolution

In our CVPR paper “Gyro-based Multi-Image Deconvolution for Removing Handshake Blur” [4], we proposed a multi-image deblurring system that uses gyroscope data for camera motion estimation. Although our method effectively removes handshake blur in static regions, moving objects and over-exposed regions are not handled properly. These regions are considered as outliers because they do not follow our image formation model. Moving objects may suffer from excessive blur caused by the object motion occurred during the entire capture time. In addition, pixel values in highlights may be clipped due to limited dynamic range of image sensor, and often cause artifacts after deconvolution. Patch-based denoising method merges similar image patches to effectively reduce noise [2][3]. When multiple images are available, these methods reduce noise in a reference image using image patches from all available images. One advantage of this type of denoiser is that outliers are handled better than with multi-image deconvolution. That is, although moving objects remain as blurry as in the reference image, the amount of motion blur is less than when multi-image deconvolution is used. In addition, while overexposed regions may also contain some blur, they will not suffer from noticeable artifacts. In this technical report, we propose an additional image blending step that follows non-blind multi-image deconvolution. Our goal is not to handle outliers in a physically based method. Instead, we hide possible deconvolution artifacts. First, we detect outliers in the image obtained from multi-image deconvolution. Then, the pixel values around outliers are blended with the result of patch-based denoising. As a result, the blended image contains sharp details in static regions, while moving objects and over-exposed regions are expressed naturally without excessive blur.